Dynamic Resource Allocation in the Cloud Computing Using Nephele’s Architecture

In recent years ad-hoc parallel data processing has emerged to be one of the most important applications for Infrastructure-as-aService (IaaS). Major Cloud computing companies have started to integrate frameworks for parallel data processing in their product portfolio, making it easy for customers to access these services and to deploy their programs. However, the processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud. Consequently, the allocated compute resources may be inadequate for big parts of the submitted job and unnecessarily increase processing time and cost. Nephel’s architecture offers for efficient parallel data processing in clouds. It is the first data processing framework for the dynamic resource allocation offered by today’s IaaS clouds for both, task scheduling and execution. Particu lar tasks of a processing job can be assigned to different types of virtual machines which are automatically instantiated and terminated during the job execution.